Conventionally, there is a technology for generating an image of a virtual world focused in accordance with a distance from a viewpoint (depth). The expression “image focused in accordance with a distance from a viewpoint” refers to an image in which an object located closer to the focal point of a virtual camera in the depth direction (viewing direction) is shown with a clear outline and an object located farther from the focal point of the virtual camera in the depth direction is shown with a blurred outline. Such an image can represent the distance of different parts of the image from the viewpoint more realistically.
Patent document 1 (Japanese Laid-Open Patent Publication No. 2001-175884) describes an image generation system for generating an image focused as described above. According to this image generation system, an original image and a blurred image are generated, and a focused image is generated by synthesizing the original image and the blurred image based on an α value which is set for each pixel. The α value is in the range of 0≦α≦1, and represents the synthesizing ratio of the original image and the blurred image. In the above-described image generation system, the α value is set in accordance with a depth value of each pixel of the original image. Since the synthesizing ratio (α value) of the original image and the blurred image changes in accordance with the depth value of each pixel, an image focused in accordance with the distance from the viewpoint can be generated.
FIG. 23 shows an exemplary focused image generated by such a conventional method. In FIG. 23, disc-shaped objects 91 through 93 are located in a virtual space. The object 91 is located closest to the viewpoint of the virtual camera, and the object 93 is located farthest from the viewpoint of the virtual camera (at the deepest position of the three objects). In FIG. 23, it is assumed that the focal point of the virtual camera is at the position of the object 92. For the pixels of the object 92 at the focal point, the α value is set to 0. Therefore, the original image is reflected and the object 92 is shown as being well focused (with a clear outline). For the objects 91 and 93 away from the focal point, the α value is set to a value closer to 1. Therefore, the blurred image is reflected to a high degree and the objects 91 and 93 are shown unfocused (with a blurred outline).
According to the method described in patent document 1, the α value is set to 0 for the pixels of the object 92 located at the focal point. For the pixels in the vicinity of a border L between the objects 91 and 92 shown overlapping each other, the α value is set to 0 for the pixels of the object 92 and the original image is reflected. Therefore, as shown in FIG. 23, the object 91 is shown with a clear outline along the border L, although the object 91 would be seen more realistically with a blurred outline along the border L. As a result, the object 91 is shown unnatural.
According to the method of patent document 1, the α value is simply set in accordance with the depth value of each pixel. For this reason, where pixels having significantly different depth values are closely adjacent to each other, the border between the focused pixels and the unfocused pixels are presented clearly. The resultant image appears unnatural.
Therefore, certain example embodiments provide an image processing program and an image processing apparatus for generating a more realistic image.
The reference numerals, additional descriptions and the like in parentheses in this section of the specification indicate the correspondence with the embodiments described later for easier understanding of certain example embodiments, and are not limiting in any way.
A first aspect of certain example embodiments is directed to a computer-readable storage medium having stored thereon an image processing program (game program 60) to be executed by a computer (CPU 10 and/or GPU 11b) of an image processing apparatus (game apparatus 3) for generating an image. The image processing program causes the computer to execute an α value setting step (S4), an increasing step (S5, S6), and a synthesis step (S7). The α value setting step sets an α value (α information) of each pixel of a reference image (FIG. 8) having a depth value (Z value) for each pixel, the α value being set in accordance with the depth value of the respective pixel. The increasing step increases the α value which is set for a pixel having a smaller α value among two adjacent pixels (FIG. 16 or FIG. 21). The synthesis step synthesizes the reference image and a blurred image (FIG. 9) corresponding to the reference image based on the α value which is set for each pixel after being processed by the increasing step.
According to a second aspect of certain example embodiments, the increasing step may be performed only where the two adjacent pixels have an α value difference of a predetermined value or greater.
According to a third aspect of certain example embodiments, in the α value setting step, the computer may set the α value such that a ratio of the reference image is higher for a pixel having a depth value closer to a predetermined reference value.
According to a fourth aspect of certain example embodiments, in the α value setting step, the computer may set a two-dimensional vector for each pixel by setting the α value as a first component value (n component value) of the two-dimensional vector for a pixel having a depth value smaller than the predetermined reference value, and by setting the α value as a second component value (f component value) of the two-dimensional vector for a pixel having a depth value larger than the predetermined reference value (FIG. 14). In this case, the increasing step comprises a smoothing step (S5) and an addition step (S6). The smoothing step smoothes a value of the first component of the two-dimensional vector which is set for each pixel. The addition step sets a sum of the first component value and the second component value of the two-dimensional vector as the α value for each pixel. In the synthesis step, the computer synthesizes the reference image and the blurred image based on the α value which is set for each pixel after being processed with the addition step.
According to a fifth aspect of certain example embodiments, in the α value setting step, the computer may set 0 as the second component value of the two-dimensional vector for a pixel having a depth value smaller than the predetermined reference value, and may set 0 as the first component value of the two-dimensional vector for a pixel having a depth value larger than the predetermined reference value.
According to a sixth aspect of certain example embodiments, in the increasing step, the computer may process only the pixels having a depth value which is equal to or smaller than a predetermined value.
According to a seventh aspect of certain example embodiments, in the increasing step, the computer may smooth the α value of each pixel.
According to an eighth aspect of certain example embodiments, in the increasing step, the computer may select a pixel having an α value which is smaller than the α value of an adjacent pixel and is different from the α value of the adjacent pixel by a predetermined value or greater, and may increase the α value of the selected pixel.
According to a ninth aspect of certain example embodiments, in the increasing step, the computer may select a pixel having an α value which is smaller than the α value of an adjacent pixel and is equal to or smaller than a predetermined value, and may increase the α value of the selected pixel.
According to a tenth aspect of certain example embodiments, the image processing program may cause the computer to further execute a blurred image generation step (S3) of generating the blurred image by smoothing a color value of each pixel of the reference image.
An eleventh aspect of certain example embodiments is directed to a computer-readable storage medium having stored thereon an image processing program (game program 60) to be executed by a computer (CPU 10 and/or GPU 11b) of an image processing apparatus (game apparatus 3) for generating an image. The image processing program causes the computer to execute a depth value correction step (S11), an α value setting step (S12), and a synthesis step (S7). The depth value correction step corrects a pixel having a depth value closer to a predetermined reference value among two adjacent pixels of a reference image (FIG. 8) having a depth value for each pixel, the pixel being corrected such that the depth value thereof becomes closer to the depth value of the other pixel. The α value setting step sets an α value of each pixel in accordance with the depth value of the respective pixel after being processed by the depth value correction step. The synthesis step synthesizes the reference image and a blurred image corresponding to the reference image based on the α value which is set for each pixel.
According to a twelfth aspect of certain example embodiments, the depth value correction step may be performed only where the two adjacent pixels have a depth value difference of a predetermined value or greater.
According to a thirteenth aspect of certain example embodiments, in the α value setting step, the computer may set the α value such that as a ratio of the reference image is higher for a pixel having a depth value closer to the predetermined reference value.
According to a fourteenth aspect of certain example embodiments, the depth value correction step may be performed on only the pixels having a depth value which is equal to or smaller than the predetermined value.
Certain example embodiments may be provided in the form of an image processing apparatus having the equivalent functions of an image processing apparatus for executing the steps in the first through fourteenth aspects.
According to the first aspect, for a pixel having a smaller α value among two adjacent pixels which have an α value difference of a predetermined value or greater, the α value which is set for the pixel is increased in accordance with the depth value. Thus, an image can be generated with the outline of a border portion between the two pixels being blurred. Therefore, an unnatural image in which only a part of the outline of an object is clear is prevented from being generated, and a more realistic image can be generated.
According to the second aspect, the α value correction (increase) is performed only when the α value difference is a predetermined value or greater. Thus, the pixels which need to be corrected can be corrected with certainty.
According to the third aspect, an object corresponding to a pixel having a depth value closer to a reference value is shown with a clear outline, whereas an object corresponding to a pixel having a depth value farther from the reference value is shown with a blurred outline. Thus, an image with realistic focusing can be generated.
According to the fourth aspect, the α value correction (increase) is performed at least on a pixel having a depth value smaller than the reference value. Therefore, the problem of the conventional technology that the border between a focused object and an unfocused object is clearly presented is solved.
According to the fifth aspect, the α value correction (increase) is performed only on a pixel having a depth value smaller than the reference value. A pixel having a depth value larger than the reference value is not a target of correction. Therefore, an image, in which the border between a focused object and an object located closer to the viewpoint than the focused object is blurred and the border between the focused object and an object located farther from the viewpoint than the focused object is clear (FIG. 10), is generated. Thus, an image with more realistic focusing can be generated.
According to the sixth aspect, the α value correction (increase) is performed only on a pixel having a depth value smaller than the reference value. A pixel having a depth value larger than the reference value is not a target of correction. Therefore, by setting the reference value to, for example, a predetermined value, an image, in which the border between a focused object and an object located closer to the viewpoint than the focused object is blurred and the border between the focused object and an object located farther from the viewpoint than the focused object is clear (FIG. 10), is generated. Thus, an image with more realistic focusing can be generated.
According to the seventh aspect, smoothing is performed on the α value of each pixel. Thus, for a pixel, the α value of which is to be increased (i.e., for a pixel having a smaller α value among two adjacent pixels which have an α value difference of a predetermined value or greater), the α value can be easily increased.
According to the eighth aspect, a pixel having an α value which is smaller than the α value of an adjacent pixel and is different from the α value of the adjacent pixel by a predetermined value or greater is selected. Thus, the pixel, the α value of which is to be increased, can be specified with certainty. Therefore, an image with realistic focusing can be generated without fail.
According to the ninth aspect, a pixel having an α value which is smaller than the α value of an adjacent pixel and is a predetermined value or smaller is selected. Thus, the pixel, the α value of which is to be increased, can be specified with certainty. Therefore, an image with realistic focusing can be generated without fail.
According to the tenth aspect, a blurred image can be easily generated from the reference image.
According to the eleventh aspect, for a pixel having a depth value closer to a predetermined reference value among two adjacent pixels which have a depth value difference of a predetermined value or greater, the depth α value is corrected so as to be closer to the depth value of the other pixel. The α value is set in accordance with the post-correction depth value. Therefore, substantially the same effect as that of the first aspect of correcting the α value is provided. Namely, an image can be generated with the outline of the border between the two pixels being blurred. Thus, a more realistic image can be generated.
According to the twelfth aspect, the depth value is corrected only when the depth value difference between two adjacent pixels is a predetermined value or greater. Thus, the pixels which need to be corrected can be corrected with certainty.
According to the thirteenth aspect, an object corresponding to a pixel having a depth value closer to a reference value is shown with a clear outline, whereas an object corresponding to a pixel having a depth value farther from the reference value is shown with a blurred outline. Thus, an image with realistic focusing can be generated.
According to the fourteenth aspect, the depth value correction is performed only on a pixel having a depth value smaller than the reference value. A pixel having a depth value larger than the reference value is not a target of correction. Therefore, by setting the reference value to, for example, a predetermined value, an image, in which the border between a focused object and an object located closer to the viewpoint than the focused object is blurred and the border between the focused object and an object located farther from the viewpoint than the focused object is clear (FIG. 10), is generated. Thus, an image with more realistic focusing can be generated.
These and other objects, features, aspects and advantages of certain example embodiments will become more apparent from the following detailed description of certain example embodiments when taken in conjunction with the accompanying drawings.